remotemanager.dataset.runner module

Main Runner object. This object handles data I/O and metadata for a single run instance

remotemanager.dataset.runner.format_time(t: datetime) → str[source]

Format the datetime object into a dict key

Parameters:

t (datetime.time) – time object to be formatted to string

Returns:

formatted time

Return type:

(str)

class remotemanager.dataset.runner.Runner(arguments: dict, parent, self_id: str, extra_files_send: list | str | None = None, extra_files_recv: list | str | None = None, verbose: None | int | bool | Verbosity = None, extra: str | None = None, **run_args)[source]

The Runner class stores any info pertaining to this specific run. E.g. Arguments, result, run status, files, etc.

Warning

Interacting with this object directly could cause unstable behaviour. It is best to allow Dataset to handle the runners. If you require a single run, you should create a Dataset and append just that one run.

property verbose: Verbosity

Verbose property

property database: Database

Returns the parent Dataset’s database

Returns:

Database

property parent

Returns the parent Dataset object

property identifier: str

Returns a unique identifier for this, also used in the names of Runner created files.

property serialiser

Returns the parent Serialiser object

property uuid: str

The uuid of this runner

property short_uuid: str

A short uuid for filenames

property id: str

Returns this Runner’s current ID

property name: str

Returns this Runner’s name

property runfile: TrackedFile

Filename of the python runfile

property jobscript: TrackedFile

Filename of the run script

property resultfile: TrackedFile

Result file name

property errorfile: TrackedFile

File tracker for error dumpfile

property local_dir: str

Local staging directory

property remote_dir: str

Target directory on the remote for transports

property run_path: str | None

Intended running directory. If not set, uses remote_dir

Note

If both remote_dir and run_dir are set, the files will be transferred to remote_dir, and then executed within run_dir

property run_dir: str | None

Intended running directory. If not set, uses remote_dir

Note

If both remote_dir and run_dir are set, the files will be transferred to remote_dir, and then executed within run_dir

property derived_run_args: dict

Returns the base run args.

Returns:

_run_args

set_run_arg(key: str, val) → None[source]

Set a single run arg key to val

Parameters:

  • key – name to set

  • val – value to set to

Returns:

None

set_run_args(keys: list, vals: list) → None[source]

Set a list of keys to `vals

Note

List lengths must be the same

Parameters:

  • keys – list of keys to set

  • vals – list of vals to set to

update_run_args(d: dict) → None[source]

Update current global run args with a dictionary d

Parameters:

d – dict of new args

property args: dict

Arguments for the function

property extra_files_send: list

Returns the list of extra files to be sent

property extra_files_recv: list

Returns the list of extra files to be retrieved

property history: Dict[str, str]

Sorted state history of this runner

Returns:

A dictionary mapping timestamps to log messages Note that timestamps have a /{index} suffix to indicate sub-second precision. For example: “2025-01-20 10:00:00/0” is the first log message occuring at that time. Following events will be listed as /1, /2, until the next second.

Return type:

Dict[str, str]

property status_list: list

Returns a list of status updates

insert_history(t: datetime | int | float, newstate: str, force: bool = False) → bool[source]

Insert a state into this runner’s history

Parameters:

  • t (datetime.time) – time this state change occurred

  • newstate (str) – status to update

  • force (bool) – skips checks if True

property state: RunnerState

Returns the most recent runner state

set_state(newstate: str, value: int = None, state_time: datetime | int | float | None = None, force: bool = False, check_state: bool = True) → None[source]

Update the state and store within the runner history

Added in version 0.9.3: now checks the current state before setting, won’t duplicate states unless force=True

Parameters:

  • newstate – state to set to

  • value – exclusive to the “copied” state, allows setting of the value

  • state_time – set the state at this time, rather than now

  • force – skip currentstate checking if True

  • check_state – raises a ValueError if True and newtate is not a valid state

property last_submitted: int

Return the timestamp of the last submission time. If no submission has been made yet, returns -1.

last_state_time(state: str) → int[source]

Return the timestamp of the last time the state was changed. If no state change has been made yet, returns -1.

property submitter: str

Return the submitter of this runner.

generate_runline(remote_dir: str | None = None, child: bool = False) → str[source]

Generates a runline for this runner

Parameters:

  • submitter – submitter for this run

  • remote_dir – override the remote directory

  • child – Used by Dependencies, if this runner is a child, we should skip any path modifications

stage(extra_files_send: list = None, extra_files_recv: list = None, repo: str = None, extra: str = None, parent_check: str = '', child_submit: list = None, force_ignores_success: bool = False, verbose: None | int | bool | Verbosity = None, **run_args) → bool[source]

Prepare this runner for a run by creating files in the local dir

Parameters:

  • extra_files_send – list of extra files to send

  • extra_files_recv – list of extra files to receive

  • repo (str) – override the repo target

  • extra (str) – extra lines to append to jobscript. This goes _last_.

  • parent_check (str) – INTERNAL, extra string to check that the parent result exists

  • child_submit (list) – INTERNAL, list of extra lines to submit children

  • force_ignores_success (bool) – If True, force takes priority over is_success check

  • verbose – local verbosity

  • run_args – temporary run args for this run

Returns:

True if runner is ready

Return type:

bool

generate_jobscript(parent_check: str, child_submit: list, extra: str) → None[source]

Generates and writes the jobscript for this runner.

Parameters:

  • parent_check (str) – Internal argument, the bash code to check for the parent result

  • child_submit (list) – Internal argument, the bash code to submit the child

  • extra (str) – Any extra content for the jobscript

Returns:

None

stage_python(repo: str) → None[source]

Stage a “python” based run. Generates the python runfile.

Parameters:

repo (str) – Remote relative path to the respository

stage_script(script: Script)[source]

Stage a “script” based run. Generates the python runfile.

Parameters:

script (Script) – Script entity to parameterise.

stage_none(extra: str) → None[source]

Fallback run mode when python or script are not used.

Parameters:

extra (str) – Extra args

assess_run(run_args: dict, force_ignores_success: bool = False, verbose: None | int | bool | Verbosity = None) → bool[source]

Check whether this runner should be running.

If force is True we always run

If skip is True, we have to check if a run is ongoing, or a result exists

Parameters:

  • quiet – Do not print status if True

  • run_args – Temporary args specific to this run instance

  • force_ignores_success (bool) – If True, force takes priority over is_success check

  • verbose – local verbosity

Returns:

True if runner has the green light

Return type:

bool

property is_finished: bool | None

Returns True if this runner has finished

(None if the runner has not yet been submitted)

property is_success: bool

Returns True if this runner is considered to have succeeded

property is_failed: bool | None

Returns True if this runner is considered to have failed

(None if incomplete)

read_local_files(force: bool = False) → None[source]

Reads all local files attached to this Runner.

Fills out the resulting attributes (result, error, state, etc.)

Returns:

None

verify_local_files() → bool[source]

Check the existence of local files on disk

Returns:

True if everything is okay

Return type:

(bool)

property result

Returns the result attribute, if available

property error

Error (If one exists)

property full_error: str | None

Reads the error file, returning the full error

Returns:

str

clear_result(wipe: bool = True) → None[source]

Clear the results properties and set the state to “reset”, which blocks some functions until the runner is rerun

Parameters:

wipe – Additionally deletes the local files if True. Default True

Returns:

None

run(*args, **kwargs) → None[source]

Run a single runner. See Dataset.run() for args.

This function is inefficient and should not be used in a general workflow

class remotemanager.dataset.runner.RunnerFailedError(message: str)[source]

Temporary “exception” to be passed in lieu of a missing result due to a failure.

Parameters:

message – error message